12 research outputs found
A Simple Derivation of the Refined Sphere Packing Bound Under Certain Symmetry Hypotheses
A judicious application of the Berry-Esseen theorem via suitable Augustin
information measures is demonstrated to be sufficient for deriving the sphere
packing bound with a prefactor that is
for all codes on certain
families of channels -- including the Gaussian channels and the non-stationary
Renyi symmetric channels -- and for the constant composition codes on
stationary memoryless channels. The resulting non-asymptotic bounds have
definite approximation error terms. As a preliminary result that might be of
interest on its own, the trade-off between type I and type II error
probabilities in the hypothesis testing problem with (possibly non-stationary)
independent samples is determined up to some multiplicative constants, assuming
that the probabilities of both types of error are decaying exponentially with
the number of samples, using the Berry-Esseen theorem.Comment: 20 page
The Sphere Packing Bound for DSPCs with Feedback a la Augustin
Establishing the sphere packing bound for block codes on the discrete
stationary product channels with feedback ---which are commonly called the
discrete memoryless channels with feedback--- was considered to be an open
problem until recently, notwithstanding the proof sketch provided by Augustin
in 1978. A complete proof following Augustin's proof sketch is presented, to
demonstrate its adequacy and to draw attention to two novel ideas it employs.
These novel ideas (i.e., the Augustin's averaging and the use of subblocks) are
likely to be applicable in other communication problems for establishing
impossibility results.Comment: 12 pages, 2 figure
The Sphere Packing Bound For Memoryless Channels
Sphere packing bounds (SPBs) ---with prefactors that are polynomial in the
block length--- are derived for codes on two families of memoryless channels
using Augustin's method: (possibly non-stationary) memoryless channels with
(possibly multiple) additive cost constraints and stationary memoryless
channels with convex constraints on the composition (i.e. empirical
distribution, type) of the input codewords. A variant of Gallager's bound is
derived in order to show that these sphere packing bounds are tight in terms of
the exponential decay rate of the error probability with the block length under
mild hypotheses.Comment: 29 page
The Augustin Capacity and Center
For any channel, the existence of a unique Augustin mean is established for
any positive order and probability mass function on the input set. The Augustin
mean is shown to be the unique fixed point of an operator defined in terms of
the order and the input distribution. The Augustin information is shown to be
continuously differentiable in the order. For any channel and convex constraint
set with finite Augustin capacity, the existence of a unique Augustin center
and the associated van Erven-Harremoes bound are established. The
Augustin-Legendre (A-L) information, capacity, center, and radius are
introduced and the latter three are proved to be equal to the corresponding
Renyi-Gallager quantities. The equality of the A-L capacity to the A-L radius
for arbitrary channels and the existence of a unique A-L center for channels
with finite A-L capacity are established. For all interior points of the
feasible set of cost constraints, the cost constrained Augustin capacity and
center are expressed in terms of the A-L capacity and center. Certain shift
invariant families of probabilities and certain Gaussian channels are analyzed
as examples.Comment: 59 page
The Sphere Packing Bound via Augustin's Method
A sphere packing bound (SPB) with a prefactor that is polynomial in the block
length is established for codes on a length product channel
assuming that the maximum order Renyi capacity among the component
channels, i.e. , is . The
reliability function of the discrete stationary product channels with feedback
is bounded from above by the sphere packing exponent. Both results are proved
by first establishing a non-asymptotic SPB. The latter result continues to hold
under a milder stationarity hypothesis.Comment: 30 pages. An error in the statement of Lemma 2 is corrected. The
change is inconsequential for the rest of the pape
Error-and-Erasure Decoding for Block Codes with Feedback
Inner and outer bounds are derived on the optimal performance of fixed length
block codes on discrete memoryless channels with feedback and
errors-and-erasures decoding. First an inner bound is derived using a two phase
encoding scheme with communication and control phases together with the optimal
decoding rule for the given encoding scheme, among decoding rules that can be
represented in terms of pairwise comparisons between the messages. Then an
outer bound is derived using a generalization of the straight-line bound to
errors-and-erasures decoders and the optimal error exponent trade off of a
feedback encoder with two messages. In addition upper and lower bounds are
derived, for the optimal erasure exponent of error free block codes in terms of
the rate. Finally we present a proof of the fact that the optimal trade off
between error exponents of a two message code does not increase with feedback
on DMCs.Comment: 33 pages, 1 figure
A Simple Converse of Burnashev's Reliability
In a remarkable paper published in 1976, Burnashev determined the reliability
function of variable-length block codes over discrete memoryless channels with
feedback. Subsequently, an alternative achievability proof was obtained by
Yamamoto and Itoh via a particularly simple and instructive scheme. Their idea
is to alternate between a communication and a confirmation phase until the
receiver detects the codeword used by the sender to acknowledge that the
message is correct. We provide a converse that parallels the Yamamoto-Itoh
achievability construction. Besides being simpler than the original, the
proposed converse suggests that a communication and a confirmation phase are
implicit in any scheme for which the probability of error decreases with the
largest possible exponent. The proposed converse also makes it intuitively
clear why the terms that appear in Burnashev's exponent are necessary.Comment: 10 pages, 1 figure, updated missing referenc
Refined Strong Converse for the Constant Composition Codes
A strong converse bound for constant composition codes of the form
is
established using the Berry-Esseen theorem through the concepts of Augustin
information and Augustin mean, where is a constant determined by the
channel , the composition , and the rate , i.e., does not depend
on the block length .Comment: 7 page
Bit-wise Unequal Error Protection for Variable Length Block Codes with Feedback
The bit-wise unequal error protection problem, for the case when the number
of groups of bits is fixed, is considered for variable length block
codes with feedback. An encoding scheme based on fixed length block codes with
erasures is used to establish inner bounds to the achievable performance for
finite expected decoding time. A new technique for bounding the performance of
variable length block codes is used to establish outer bounds to the
performance for a given expected decoding time. The inner and the outer bounds
match one another asymptotically and characterize the achievable region of
rate-exponent vectors, completely. The single message message-wise unequal
error protection problem for variable length block codes with feedback is also
solved as a necessary step on the way.Comment: 41 pages, 3 figure